Thanks for the fix. Can you add a unit test that fails without this fix, but passes with this fix?

Hi @TobyRoseman I've just discovered that the root cause of this issue is the function ModelDebugger.step which applies check_intermediate_output explicitly to all original model outputs as seen here:

Hence, I believe rather than changing check_intermediate_output which is correctly expecting only np.ndarray, it's better to adjust ModelDebugger.step so that it doesn't call check_intermediate_output on the final model outputs. More concretely I'm thinking that we can add a new argument intermediate_outputs_only=False which can be set to True to skip the above block of code, whilst retaining the original functionality of step by default.

What do you think?

Thanks @Zerui18 for looking into this. I'm not very familiar with that part of the code. I think @yixingli-apple or @junpeiz should be able to help here.

Hi, is there any updates on this?

Hi @Zerui18 ,

I agree with you we should make a fix in _model_debugger.py instead.
When we're doing calibration, we should exclude any output tensors. There’s no need to add a new argument intermediate_outputs_only=False, but simply exclude all output tensors (in intermediate_outputs) should be good.

We just released coremltools 8.1, and the part of code have been changed. Can you please rebase and create a PR based on that?

Hi @yixingli-apple , I've created a new PR #2405 based on your suggestion.